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The procedures, major findings, and conclusions of 
the Adolescent Reasoning Project are summarized in this final report 
to the National Science Foundation. Eleven different experiments were 
conducted to investigate the role ,of naturalistic and laboratbry task 
consent on scientific reasoning. Participants involved 1500 seventh, 
ninth, eleventh; and twelfth graders from three school districts 
varying in socioeconomic status, proximity to in urbpn area, and 
sophistication of course .of firings ip math &nd* science. The appendix 
describes the purpose of eaci^experiment and ^indicates the project 
report publication number that summarizes the Results of the 
findings. The body of the reporKpresents the integrated results of 
all the experiments and what tAg^^ject staff thinks the results 
mean. Major findings are* (1) "Wudents' respond to naturalistic and 
laboratory tasks differently; {£) differences in expectations about 
the task variables account for much of the variance between 
laboratory and naturalistic tasks; "(3) subjects learn what they ate 
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in the studies. 
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Pinal Report: Evaluation of Scientific Reasoning Ability in 
z * Naturalistic and Laboratory Tasks 

Overview 

This report sunimarizes the work of the. Adolescent Reasoning Project (ARP) 
conducted for NSP-RISE grant SED 77 18914. entitled "Evaluation of Scientific 
Reasoning Ability in Naturalistic and Laboratory Tasks/ 1 Project work is 
summarized in two ways. The sections called "Major Findings 11 and "Conclusions/ 1 
integrate our results and indicate what we think they mean. The appendix 
called "Conf^BBF' Proposed Research" describes the subjects and the experiments 
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j \ Major Findings • / 

? . » 

* - 

The Adolescent Reasoning Project (ARP) was funded by the National Science 

foundation (RISE-SED 77-18914) to instigate the role of naturalistic and 

* » * • 

laboratory ta^k content on scientific reasoning.^ Most science instruction 

uses laboratory problems. We set out to determine whether inherent differ- 

ences between laboratory and naturalistic problems would limit the impact of 

classroom science instruction on naturalistic problem solving. «* 

We conducted eleven experiments, administering over 4,000 hours of group \^ 

and individual t^ests^ to over 1,500 subjects. Over 35 articles and over 15 ^ 

f J r 

meeting presentations summarize our findings, r 

Our major f indings'%re: a) students respond to naturalistic and labor- 
atory tasks differently, b). differences in^ expectations about the task vari- 
ables account for much of the variance between laboratory and naturalistic 
t^sks, c) subjects leam^what they are taught about controlling variables 
and about expectations, but do not readily generalize their training to new . 
situations, d) scientific reasoning overlaps extensively with general ability, 
but also overlaps with an aspect of field-dependence-independence -identified . 

in our studie§. 
* * 

These results suggest that subjects of ten fail)scientif ic reasoning tasks ' 

/ 

by considering oily the variables >hey think are important. Subjects ignore 
variables mentioned by others, .or present in,thte experimental situation which 
they consider unimportant. Instruction can help subjects develop better expec- 
tations about the variables and consequently perform better., 

Scientific reasoning appears to be composed of general ability and an 
aspect of field-dependence- independence which we identified galled Familiar 

Field. Scientific reasoning may be difficult to teach just as genera^ability 

■ ■ 

is difficult to mddify. FamiliariField appears to measure the ability to 
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, f * -2- 

'select from among fjtailiar competing strategies., Scientific reasoning instruc 

* 

tion might focus on techniques to help students select from among plausible* 
alternative strategies. » • - ♦ 

,Jjfe elaborate our major 'findings in the next sections. Implications of 
\ 

the$e 4 findings are also presented. , ^> 

| Naturalistic and Laboratory Tasks 

' Our research, revealed overall differences in reasoning^' Qn laboratory and 

naturalistic tasks (e.g., Pulos. & Linn, ARP-8; Clement & Pulos, ARP-29 ) . Once 
• .general effects were established, specific explanations for these effects wete 

sought. ^ k ^ 

"X 

Role of Expectations in Reasoiling Performance » 
; :* 

We found that subjects' expectations about the variables in laboratory 
arid naturalistic tasks differed 'and these expectations accounted for much of 
the difference # in performance. By Expectations, we mean beliefs concerning , 
which factors influence problem outcomes and how that* influence occurs. For 
example, mafty adolescent reasoners inaccurately expect the weight of the bob 
to jii^jFluence the oscillation o*f the pendulum. 'Thus, ^e'asoners 1 expectations 
about the pendulum influence performance, i ' 

Qur emphasis pn expeqtationS contrasts with Piaget's emphasis on logical 
strategies (e.g., Linn ? ARP-26) . Our research ^xam^ned how expectations influ 
ence 1 qomplex reasoning, taslc^ such >as those studied by Inhelder and Piaget 
(1958) . /m We are concerned ^ith when an available logical strategy is applied 
and with why available logical^ strategies ar,e not alwaya applied. 

- Our contention is that errors in reasoning often reflect expectation- 
based mi&understaftdings. Subjects fail to control for the^material of the 



1 • ^ ^ 

froject publications are referred to by ARP-numb^rs. These refer to 

project reports listed by 'number in .the Afolescent Reasoning Project List o'f 
O , .Publication s which is attached/ ■ ~ — ■ * 
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rod 'in Biding Rods because they expect "all metal rods bend the same". We 
have identified specific expectation-based errors* in reasoning and used these, 
to gain a precise understanding of reasoning performance, (See Linn, ARP-28, 

for a mbre deta*£i^d ^discussion of this point.) 

* , * * / - I 

In contrast, Piagtetian research and most replications confound task expec 

tations with. strategy , e.g., using a balance beam to measure proportions and 
bending rods to measure controlling variables. In addition, studies of ^scien- 
ti^c reasoning have, by and large, b^en conducted using tasks from physics, 
thus ignoring possible expectations associated with physics knowledge,. Expec- 
tations are real, but not frequently investigated in Piagetian-based research. 
t - 
In these studies we refer to the variables' that the subjects consider in 

* » * * 

solving the problems as the subjects' expected variables . The variables 

included in the task <are called the experimenter's comprehensive variables . 

« 

Responding to evidence (e.g., Novell, 1961)" that formal Reasoning was 
less prevalent than he anticipated, Piaget (1972) suggested that individuals 
might exhibit formal reasoning in their area of expertise. Piaget suggested 
that, for a given problem, experts in the £ield would reason better than 
novices. Applying Piaget' s notions about expertise to our ideas about expecta- 
tion-based influences on. reasoning, we hypothesized . that eaqterts would expect 
more variables to influence the outcome of a problem in their area of expertis 

than would novices. We investigated this hypothesis in a series of studies! 

* * 

Pulos and Linn (ARP-8) investigated; the influence of expertise^ reason- 
in g* An, ecologically valid comparison was possible. One group of sub^e^is ' 
were experts in fishing because they lived adjacent to a river delta; for them, 
variables influencing catching fish were familiar. Another group were expert^ 
in solving physics problems because they hVd trflcen experiential science courses 
for them, variables influencing a physics task like Bending Rods were familiar. 
Both groups responded to controlling variables tasks about fishing and tasks ' 
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about physics. ' For each group,. one task had? familiar variable's. In addition, 

both groups took 'a tKird taskwhich was parallel to the other two but* l)ad arti- 1 

» , * 

ficial content involving space people* pn a fictitious planetf. Variables iji 

this task were unfamiliar for both groups. In each task, subjects designed* 

controlled experiments. All three tasks could be solved without any expertise 

in the area. Th'ere was a significant effect for expertise on performance: 

^river delta subjects performed better on Fishing than on Bending Rods while 

experiential science subjects performed bettef on pending Rods than on Fishing. 

■ 

Thexe were no differences in performance on the artificial task. If we assume 
that expectations vary with expertise , these results suggest tihat expectations * 
influence controlling variables performance. 

Linn and Swiney (ARP-22). investigated the relationship between twelfth 
grader's expected variables* and the variables they correctly utilized on a" con- 
trolling variables task. The subjects' expected variables were measured by 
having subjects examine the' task apparatus and "name the variables they expected 
would influence the outcome. Subjects were presumed to name their expected 
variables. Subjects were then told the experimenter's comprehensive variables 
(variables the experimenter thought were -'important) . The variables correctly 
utylzed by the subject were those variables correctly controlled or investi- 
gated ih three experiments. Comparison of the* subject's expected variables and 
the variables the subject correctly utilized revealed that subjects utilize^ 
their expected variables signif icantly : more often than other variables. Sub- 
jects appeared to answer the controlling variables questions as if only their 
expected variables were important — subjects ignored the experimenter's comp£e- ** 

Linn, Clement, and Pulos (ARP-2S)) extended the findings of Linn and Swiney 

< * 

(ARP-22) using seventh, ninth, dnd eleventh graders. They hypothesised that | 
expectations would vary with task content. Naturalistic task| have content from 
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.frequently encountered situations such as determining which is- the best tooth- 
paste to btiy or. determining 'how to get the best gas mileage. . Laboratory tasks 

> * * * 

♦ generally have content from physics' such as determining what makes rods bend. 

In^he natura lis tier tafcks,, as in the labqratory tasks, subjects wfere asked 
about the design of controlled experiments ✓ 

^inn, Clement', and Pulos measured expectations and found that 

they differed for laboratory and naturalistic tasks: subjects expected to con- 
sider more laboratory task variables than Naturalistic task variables. Consis- 
tent with t^ findings of LJim and Swiney (ARP-22), subjects performed best on < 
controlling variables questions when their expected variables closely approxi- 
mated th'fc experimenter's comprehensive variables.^ The subjects performed 
better on controlling variables for laboratory than for naturalistic tasks* 
because their expected variables were more comprehensive than for naturalistic 
tasks. Expectations strongly influenced reasoning performance: 1 between 8% 
and 20% of the variance on controlling questions was; associated with content/ 

In general, it appears that subjects reason about their expected variables. 
Pulos, and Linn (ARP-8) found that subjefits reason bettdr in their area of ex£er- 
tise presumably because they know more of £he" variables. Linn and Swiney (ARP- 
22) found that subjects control their expected variables. Linn, Clement, and 
Pulos vCARP-27) found that performance on controlling variables tasks reflected 
the magnitude .of the subjects expected variables. 

Instructional, Interventions 

Many programs to encourage better Reasoning about complex math-^and sciende- 
related problems have been tried^with liiited success (see Levine & Linn, 1977; 
Linn, ARP-16). Teaching new strategies has been much less successful than teach- 
ing new ways to use existing strategies. Piaget (quoted by Hall, 1970) has * 
remarked that instruction in, the content-frrfe strategies characteristic of/ formal 
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reasoning is useless. - Many Unsuccessful studies- attest .to this assertion 
(e.g., Levine & Linn, l^R; Linn, ARP 19) . s 

In contrast, subjects have been taught? to contrpl variables on a wider 
, variety of problems (Linri, ARP-16), oar to apply proportional^ reasoning more v 

consistently ^fcurtz & Karplus, 1979). t Thus, -it is possible to differentiate 
• f between teaching/ previously unfamiliar strategies and teaching new" applica- 
tions of available strategies. We' investigated instructional inte^yentions 
designed to filter inaccurate expectation-based rule usage and therefore en- 
hance application of available 1 s trategies . / v ^ 

Of course , strategy application cani^f occur unless jt;he strategy has 
H^een acquired, instruction which combines strategy training with, applica- 
tion training succeeds more* consistently than either one by itself (e.g., 
Linn, ARP-16, AftP-19; Kuhn & Angelev, 4976; Linn, 'Chen, & Thier, 1979). 

Methods for .teaching strategy application are not well established. 
Cronbach and Snow (1977), in their book on aptitude treatment-interactions, ; 
reporlj successful instances of teaching strategy application which 'they N call 
"tuning." Tuning is a form of instruction which helps subjects recognize 
when to'wse an avs^lable complex reasoning § strategy. Examples ,of procedures 
which can be classified as tuning include: a) practice items r b) instruc- 
tions which note the similarity between new items and familiar items/ e.g., 
saying "These litems involve computation or proportions", c) instructions 
'which specify what about the problem i£ most important, -e.g., saying, "In 
*"these ^.tems, first figure out what variable is being' investigated, then 
figure out which variables need to be kept the same." 

Results of our studies to change expectation -based rule usage were mixed 

/ • 

Lfnn and .Delucchi (ARP-32) attempted to increase the subjects' expected varia- 
bles. Their "variables training" explained that people only control' variables 
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they expect will influence the outcome, yet other variables may also'be impor 
tant. Subjects did not change their expectations. Subjects probably need 

r 

more direct evidence that their expectations are incorrect before they Will 

change their expectations* ' „ 

« * 3 ' 

/ In a^ series oi studies of the pendulum (Linn, 1977; Pulos & Linn, ARP-?) 
direct evidence for inaccurate expectations was given. Many subjects based 

, responses to Pendulum on expectations that the weight of the bob and the , 

• • ■ * 

height of the release point influence oscillation rate. During expectation 
training subjects conducted controlled experiments * to investigate each varia- 
ble. This, direct evidence of inaccurate expectations enhanced performance on 
the Piagetian version of the Pendulum. 

Clement; Linn-, and Pulos (*ARP-33) attempted to alter expectations about 
'''the factors which influeiice one's blood pressure. They had three conditions: 

1) no training, 2) a week-long classroom unit' on blood pressure variables, 

' t ' ' ' ' 

and,' 3) a 40-minute individually tutored'unit on strategies for controlling 

variables. They found that the 'classroom unit on* variables changed the sub- 
ject f s expected variables and that the classroom unit on variables plus the 
individual unit on strategies enhaoced the subject's abilfty to control vari- 
ables having to do with blood pressure. Performance on related questions 

■ • 
suggested that the instruction had* rather marrow effects., ( . 

v In summary, instructional interventions may fail because they do not 

clarify how strategies cart be applied to new problems. Conceivably, subject^ 

need i^ruction in both variable selection aftd strategy selection to improve 

reasoning. Research on tuning and research to alter expectations suggest, 

directions for design of instructional interventions which enhance' strategy 

application. 
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What Does Scientific Reasoning Measure ? 

We established- relationships between scientific reasoning and # coipmonly 



.studied aptitudes apd ^ftilities in order to place our tasks, in a broader- con- 
test. An aptitude model emerged from our 'work and x that of others (e.g., Snow, 
r et al./'Uqte' If* Linn & ^ll^neri, 1981^ Witkin & Goodenough, Note 2). We 
describe the model and then show how it applies to pur measures of scientific 
reasoning. •» * • 

Our research and that of others supports ^n aptitude model related to 
Horn and Cattell's (1966) conceptualization of General fluid ability, General 
crystallized ability and Spatial visualization, augm^ited 'by measures^ of field- 
dependence-independence as described ^below. Snow (1^0) has done extensivev 
research to show the value of this modeljlji research^ on' reasoning. By esta- 
blishing the relationsh;L& between these c^tanonly studied 'aptitudes and our new 
measures we cart validate our measures. * 

r ^ - , * , 

Cattell (1971) and tforn and Cattell (1966) idj^j^gd General crystallized 
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and General fluid % ability and later included SpatiaJ^BHK-ization; Spatial 
visualization was thought 'to be distinct Jrom, but correlated with, General - 
crystallized afid General- fluid ^ability for adult§.. Tests requiring identifica- 
ition of new relationships, ^such- as Letter Series, measure General fluid ability 
(Gf). Tests measuring the extent of and retrieval of overlearned information, 
such as Vocabulary, measure General crystallized ability (Gc>. Tests requiring 

* 4 

mental manipulation of figural material^ such as Paper Folding or Paper Form 
* t 

Board (e.g., French, et al*, 1962), best measure' Spatial visualization (Sv) . 
Snow et al. (Note 1) attempted tp replicate Horn #nd Cat tell v s (1966) work on 
Spatial visualization for adolescents, but <|^uld^not, separate Gf from Sv so * 
thfey labeled the combined dimension General fluid visualization (Gfv) . Thus 
Spatial visualization and General flui^ability formed a single dimension, 

■ .* - • r 



defined by tests requiting mental manipulation of figural or non-figural 
material. " ¥ < ' » 

Linn ana Kyllonen (ARP-21) clarified the relationship betweeA Gfv and 
Wftkin and Goodenough's (No'te 2) concept of f ield-dependencje-independence 
(FDI). Witkin^and Goodenongh' (Note 2) had identified two FBI dimensions. 
One, cognitive restructuring/ was measured by Embedded Figured and closely 
resembled Snow's Gfv. The other, Perception of "the Upright, v was measured* ♦ 
by ' f-Ufutt gfl and Frame^test and appeared to differ from Gfv. Linn and Kylonnen 
.(1981) combined measures Of Gfv and both aspects of FDI» Thgy identified two ' 
dimensions. One was sin^ilar to Snow's Gfv ^nd included the Cognitive Restruc- 
turing tests^and was still labeled Gfv. The other was characterized by 
Perception of the Upright but also included -the Weschler Picture Completion 
test; they labeFeU this Familiar field (Ff). The Ff dimension was hypothesized 
to measure strategy selection in familiar situations when competing strategies 
were available. Thus, two unique dimensions Jfoerged from among General fluiil 
ability, Spatial visualization and f ield-dependehce- independence — these were 
Gfv and Ff . ' 

Our aptitude model includes ^fv and Ff as well, as General crystallized 
(Gc). which remained distinct from the other two. To investigate our task 
analysis of expectation-based reasoning performance, we examined how. Gfv and 
Ff relate to reasoning performance. The relationship between our aptitude 
model and. scientific reasoning was established in Several studies (e.g., Linn, 
Pulos, and Gans, 1981, ARP-11; Linn & Swiney, ARP-22; Linn, ARB-5). All these 
studies revealed a strong overlap between scientific Reasoning and general 
ability (as measured by Gc and Gfv). 4 Several of the formal measures were 
related to Ff, Two questions emerge: a)' Is scientific reasoning a unique 
ability? and b) What is the role* of Ff in scientific reasoning? 
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\^ Some wrij^rs (e.g., Humphries & Parsons, 1980) ifeel that reasoning about 
Piagetian^tasks overlaps completely with gejaeral ability while others feel 
^^that there is unique variance w in < scientific reasoning (e.g. , Linn,\RP-9; Linn^ 
& Swiney, ARP-22). * Our studies suggest that so*e aspects of scientific reason- 
ing perfonpance reflect unique, knowledge. As suggested 'by Linn and gwiney 
(ARP-22), unique aspect! of scientific reasoning may reflect specific content 
Jcn6wledge. . f i 

<„ Many studies have phown a relationship between f ield-dependence-Indepen- 
dence and scientific reasoning including several done by our project (e.g., 
Liim & Swiney, ARP-22; Linn, ARP-5}\ Our studies refine understanding of FDI* 
by identifying the Ff dimension and showing that it measures strategy selec- 
tion in familiar situations for both controlling /Variables (Linn, ARP^5) and 
proportional reasoning (Linn &. Swiney, ARP-22). 

Strategy selection, is a connnon component of scientific reasoning. Rela^ 
tionsljips between Ff and scientific reasoning may reflect strategy selection. 
We .found that Ff only contributes to 'performance for f orraal-reasoning tasks, 
which are difficult for most .subjects,' Familiar field appears to tap an aspect 
of identification of new relationships. As scientific reasoning strategies 
develop, they come tff compete with other incorrect ^strategies. .Thus, Ff 
appears to be associated with scientific reasoning tasks when there is competi- 
tion among familiar strategies* 
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Implications S 

* 

Following the tradition established by Piaget, research on scientific 
reasoning has focused on factors emphasized by the theory rather than on e>fpec- 
fat ions about the task variables. Our research suggests that expectations 
deserve, more careful scrutiny, Our research has shown that specific expecta- 
tions, individual aptitudes, and instructional interventions can all influence 

ERIC > 14 



performance. Each of tt^ese aspects of ^cientific^ reasoning suggest directions 
for educational intervention. Researchers interested in education are. encour- 



aged to focus their efforts ontlTtSe aspects of scientific reasoning perfor- 
mance. For extended discussion of the implications of our findifigs, «ee' the 

„ • 

attached project publications (ARP 19^ 21, 28, 36). 

.. ? * 

These finding's, takfcri together, suggest that researchers should analyze 
performance at a more specific level than the Piagetian stage. Expectations 
about task variables ar$ not a part of the Piagetian stage 4 but influence 
classrpom performance. Diagnosis of specific reasoning errors based on expec- 
tations suggest how reasoners progress within stages. Aptitude differences 
amoqg learners may clarify why some individuals achieve scientific reasoning 
and others do not. Finally, design and evaluati^b of instruction focused on 



specific Reasoning errors may clarify why scientific reasoning is so difficult 

# 

to teach. 

1 * - 

V 

Expectations 

Investigation of ttie expectations about task content in scientific fceason- 
ing c&n validate and clarify Piagetian theory. Although Piaget has not focu^l 
on specific errors, our research identifies specific errors based on expecta- 
tions and gives a more precise picture of how reasoners move from one stage to 
another. Establishing the effect of expectations on reasoning may simplify 
investigation of theoretical factors'. Content-free strategy acquisition must * 
be separated from the effects of^etrntent to be investigated clearly. 

The role of expectations in formal reasoning deserves scrutiny by science 
educators because these factors are likely to have practical implications. Edu 
cay:ors. frequently use research to design curricula. Research which helps curri 
culum developers anticipate and remediate reasoning errors will enhance science 

education. As an ultimate goal, research on expectations might help teachers 

* J 

' $ 

IS 



develop skills in diagnosing errors made by^students in their classes. 

ExpectatiQr> effects clarify Research findings of pervasive inaqcurate 
physics reasoning, .'Mcbermgtt .(Note 3) reports inaccurate reasoning ^about 
acceleration; Clement (1979)^ reports inaccurate reasoning about force; 
Champagne, Klopfer, ana Anderson (1979) report inaccurate reasoning about 
mechanics. If subjects' bkve inaccurate expected variables they might Vea- 
son inaccurately because they only reason about their ^expected variables. 
In thB research reported here, subjects considered only their expected 
variables, thereby omitting '.important variables, In other situations, by 

reasoning about their expected* variables, subjects might include unimpor- 

* *» 

tant variables such as weight^as a variable in the oscillation of the pen- 
dulum. Thus, studie§Nafe^j.ti^inaccurate physics reasoning may reflect 
inaccurate expected variables, ^ 

* "Our* research alstf suggests that instruction in laboratory problems is 
unlikely to generalize to n^lralisfcic problems. Naturalistic tasks- are 
important to insure that science education fosters scientific literacy. 
Instruction using naturalistic situations would enhance tfie relevance of 
the instructional progjrani. It is essential to provide instructi0n that 
helps students evaluate Expectation? about naturally occurring problem's. 

4 

Laboratory tasks m^y not require such evaluations. Since it is apparent 
that evaluation of expectations is very important for naturalistic problem 
solving, efforts to choose problems for science instruction to illustrate 
the" role of expectation^ would enhance scientific literacy, ^ 

These results suggest that educational ^implications generated frojn 
Piagetian theory may be incomplete. Expectations require more emphasis. 
Consistent with Duckworth's (1979) suggestions for younger children, vari- 
ability iri performance on^reasoning problems may not be due so much to lack 

. : ' 1 Pi 
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of developmentally baqed, Strategies, as trd lack of instruction in how to com- 
bine txpsictatioRS and strategies. 

Aptitudes " .* ' * ' 

Capitalizing on aptitude-treatment-int^ractions may enhance the effec- 
tivene^ of instruction. For -example* the Linn (AKP-5) study suggests that 
subjects who aie field dependent. should first be taught to control variables 
in problems wit hoot irrelevant information .and then specifically taught to 
identify ^and eliminate * irrelevant information. v By carefully selecting exam- 

pies* and in other" ways specifically modifying science instruction, educators 

» » 

will ultimately help students of a variety of aptitudes perform bfcttei in 
math and science. 4 



\ 
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APPENDIX ' -ik 

■v 

* Conduct of Proposed Research ' ^ * * 

* t 
i I 

I 

■ t ' * 

' Our^ research project conducted eleven 'experiments H|/plving about 15^0 
seventh, ninth, and eleventh grade subjects in three school disfrii^l. We 
administered 4000 hours of gfoup and , individual tests, analyzed the results 
and reported our findings in over 35 articles and over 15 presedtations at 

'meetings. * ^ 

■» 

Subjects - - 

Subjects were 1500 seventh, ninth, eleventh, and twelfth grade students 

from Bay Area schools. Our main sample. of 900 subjects consisted of approx- 

imately 100 students per grade for* seventh, ninth, and eleventh grade in 
» 

three schools. Each study used 90 or more students, ten ftfom each grade in 
each schqol. Schools varied in socioeconomic status, proximity to an tfrban 
area, and sophistication of course offerings in mith and science. 

Experiments 

We proposed and carried out eleven experiments. The results that we 

: y 

anticipated did not always coincide with the results that we collected. Some 
of our anticipated results did not materialize while other results fit differ- 
ent interpretations than we expected. Publications describing each experiment 
► have been produced by project personnel In this section we desdribe the 

relationship between our proposed research and our experiments. In the section 

* -t * 

called "Major Findings, and Conclusions," we give our current interpretation of 

*%\xv findings. 

'i 

^Design of Instruments 

• The project has designed, pilot tested, and validated a <wide range of 
tasks. . ^e have*devis£d or revised over .20 measures of controlling variables 



which are described in several of ouf reports (e.g., Linn, Clement, 4 & Pulos, 
ARB 29). Two of our reports focus on our controlling tasks (Linn, v ARP-l; 
Linn & Rice, ARP-2). Our measure of predicting displaced^Volume is described 
in Linn and Pulos (ARP-2?) . Our mea^tfres of abilities &hd aptitudes are 
described in Linn and Pulos 

Experiments I and II ' . - 

i Our first two experiments documented the effects of expectations on 
naturalistic content and laboratory content controlling variables tasks. By 
expectations we mean belief s. about how task/variables influence task outcome- 

■ " 7 

pWe found that expectations differed for laboratory and naturalistic tasks and 
that content accounted for between 8 and 20% of the variance. This research 
is reported in Linn, Pulos, and Cans (ARP-11) , Linn and Pulos (ARP-14) and 
Linn, Clement, and, Pulos <ARP-29). 

Experiments III and IV 

''.••/ 

Two of our studies focused on reasoning in game-lfke and informal settings, 
We found few content effects for eames with imaginary content. These- studies 
^re reported in Pulos and Linn (ARP-6) , Pulos and Linn, (ARP-7) , Pulos and Linn 
(&P-8), and- de Benedictis (ARP-23). ^ . . ' 

\ * : ' ' f . "* ' V 

Experilfets V, VI, VII. VII I 

These studies focused on factors that might Jftfluence reasoning perfor- 

% • • w f \ 

mance, including the effect o0 tlte number pf , variables in the problem (Pulos 

& Linn, ARP-31) , fhe effect of the salience of the variables (Linn'& Swiney, 

ARP-22), the -effects of preconceptions about the .variables (Linn & Pulos, r 

ARP-27), and the effect of complexity of task format (Linn & Pulos, 'ARP-25; 

Liijfc, Clement & -Pulds, ARP-29) ♦ ' . « • . • 



; ' Taken -together, these studies suggested that expectations about problem 

Variables (including salience and preconceptions)' consistently influenced • 
performance. Subjects followed clearly definable rules to generate both 

their correct and .their incorrect responses. Format effects such as the v 
t number of variables in the task,, or the ease in designing a fain experiment 

also influenced, performance. The effects of format were less dramatic than^ 

the effects of expectations. I 

' Experiments IX and X ' 7 

i ' 

These experiments were preliminary studies, of how reasoning about con- 

trolling variables could be enhanced,' Pilot work indicated the difficulty 

\ ■ ) 

.in changing expectations (Pulos & Linn, ARP-3). A-studv_of training on 
strategies versus training on expectations met with/limited success (Linn 
& Delucchi, ARP-32). A large-scale study comparing- training to alter expec- 
tations and training on the design of fair experiments suggested directions 
for future studies (Clement; Linn, & Pulos, ARP-33, de Benedictis, ARP-26). 
Two reports integrate current and« previous studies of instructional inter- 
ventions to suggest how scientific reasoning can be altered (Linn, ARP-16); 

•Linn, ARP-35) .. * 

✓ - 

Experiment XI * * . % * 

This study focused on the aptitudes and abilities required to solve com- 
plex-reasoning tasks. We investigated the field dependence-independence con- 
struct (Linn, ARP^5; Linn,' ARP-4; Llrm (, Kylonnen, ARP-21). We related 
measures of previdusly studied aptittrdes and abilities to a .variety of reason- 
ing fasks (e.g., Linn,' Pulps, &r Gans, AKP-H'; Linn &'Swiney, ARP-22; Linn & 

* r 

Pulos, ARP-25, Aptitude and ability measures clarified what our new 

tas]U measured and suggested why some tasks were f ailed. X • 



Reference Notes ' - - 
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